Neural fields, also known as coordinate-based or implicit neural representations, have shown a remarkable capability of representing, generating, and manipulating various forms of signals. For video representations, however, mapping pixel-wise coordinates to RGB colors has shown relatively low compression performance and slow convergence and inference speed. Frame-wise video representation, which maps a temporal coordinate to its entire frame, has recently emerged as an alternative method to represent videos, improving compression rates and encoding speed. While promising, it has still failed to reach the performance of state-of-the-art video compression algorithms. In this work, we propose FFNeRV, a novel method for incorporating flow information into frame-wise representations to exploit the temporal redundancy across the frames in videos inspired by the standard video codecs. Furthermore, we introduce a fully convolutional architecture, enabled by one-dimensional temporal grids, improving the continuity of spatial features. Experimental results show that FFNeRV yields the best performance for video compression and frame interpolation among the methods using frame-wise representations or neural fields. To reduce the model size even further, we devise a more compact convolutional architecture using the group and pointwise convolutions. With model compression techniques, including quantization-aware training and entropy coding, FFNeRV outperforms widely-used standard video codecs (H.264 and HEVC) and performs on par with state-of-the-art video compression algorithms.

Neural radiance fields (NeRF) have demonstrated the potential of coordinate-based neural representation (neural fields or implicit neural representation) in neural rendering. However, using a multi-layer perceptron (MLP) to represent a 3D scene or object requires enormous computational resources and time. There have been recent studies on how to reduce these computational inefficiencies by using additional data structures, such as grids or trees. Despite the promising performance, the explicit data structure necessitates a substantial amount of memory. In this work, we present a method to reduce the size without compromising the advantages of having additional data structures. In detail, we propose using the wavelet transform on grid-based neural fields. Grid-based neural fields are for fast convergence, and the wavelet transform, whose efficiency has been demonstrated in high-performance standard codecs, is to improve the parameter efficiency of grids. Furthermore, in order to achieve a higher sparsity of grid coefficients while maintaining reconstruction quality, we present a novel trainable masking approach. Experimental results demonstrate that non-spatial grid coefficients, such as wavelet coefficients, are capable of attaining a higher level of sparsity than spatial grid coefficients, resulting in a more compact representation. With our proposed mask and compression pipeline, we achieved state-of-the-art performance within a memory budget of 2 MB. Our code is available at https://github.com/daniel03c1/masked_wavelet_nerf.

神经领域已成为一种新的数据表示范式，并在各种信号表示中表现出了显着的成功。由于它们在网络参数中保留信号，因此通过发送和接收整个模型参数来传输数据传输，可以防止在许多实际情况下使用这种新兴技术。我们提出了流媒体神经场，这是一个由各种宽度的可执行子网络组成的单个模型。拟议的建筑和培训技术使一个网络能够随着时间的流逝而流式传输，并重建不同的素质和一部分信号。例如，较小的子网络会产生光滑和低频信号，而较大的子网络可以代表细节。实验结果显示了我们方法在各个域中的有效性，例如2D图像，视频和3D签名的距离函数。最后，我们证明我们提出的方法通过利用参数共享来提高培训稳定性。

场景流估计在场景之间提取点运动的场景估计正在成为许多计算机视觉任务的关键任务。但是，所有现有的估计方法仅利用单向特征，从而限制了准确性和通用性。本文使用双向流嵌入层提出了一种新颖的场景估计架构。所提出的双向层学习沿向前和向后方向的功能，从而增强了估计性能。此外，层次功能提取和翘曲可改善性能并减少计算开销。实验结果表明，拟议的架构通过在FlyingThings3D和Kitti基准测试中优于其他方法，从而实现了新的最新记录。代码可在https://github.com/cwc1260/biflow上找到。

隐式神经表示（INR）被出现为代表信号的强大范例，例如图像，视频，3D形状等。尽管它已经示出了能够表示精细细节的能力，但其效率尚未得到广泛研究数据表示。在INR中，数据以神经网络的参数的形式存储，并且通用优化算法通常不会利用信号中的空间和时间冗余。在本文中，我们建议通过明确地删除数据冗余来表示和压缩视频的新型INR方法。我们提出了跨视频帧和残差的主体剩余流场（NRFF）而不是存储原始RGB颜色，而不是存储原始RGB颜色。维护通常更光滑和更复杂的运动信息，比原始信号更少，需要更少的参数。此外，重用冗余像素值进一步提高了网络参数效率。实验结果表明，所提出的方法优于基线方法的显着边际。代码可用于https://github.com/daniel03c1/eff_video_repruseentation。

通过其高能量效率，加工存储器（PIM）阵列越来越多地用于卷积神经网络（CNN）推断。在基于PIM的CNN推断中，计算延迟和能量取决于CNN权重映射到PIM阵列的方式。最近的一项研究建议的移位和重复的内核（SDK）映射，其用一个并行窗口的单位重用输入特征映射，其与重复的内核卷积以并行获得多个输出元素。但是，现有的基于SDK的映射算法并不总是导致最小计算周期，因为它只将方形的并联窗口与整个通道映射。在本文中，我们介绍了一种名为可变窗口SDK（VW-SDK）的新型映射算法，其自适应地确定了对给定卷积层和PIM阵列的最小计算周期的并行窗口的形状。通过允许具有部分通道的矩形窗口，VW-SDK更有效地利用PIM阵列，从而进一步减少计算周期的数量。与512x512 PIM阵列和RESET-18的仿真显示，与现有的基于SDK的算法相比，VW-SDK将推理速度提高1.69倍。

The current reinforcement learning algorithm uses forward-generated trajectories to train the agent. The forward-generated trajectories give the agent little guidance, so the agent can explore as much as possible. While the appreciation of reinforcement learning comes from enough exploration, this gives the trade-off of losing sample efficiency. The sampling efficiency is an important factor that decides the performance of the algorithm. Past tasks use reward shaping techniques and changing the structure of the network to increase sample efficiency, however these methods require many steps to implement. In this work, we propose novel reverse curriculum reinforcement learning. Reverse curriculum learning starts training the agent using the backward trajectory of the episode rather than the original forward trajectory. This gives the agent a strong reward signal, so the agent can learn in a more sample-efficient manner. Moreover, our method only requires a minor change in algorithm, which is reversing the order of trajectory before training the agent. Therefore, it can be simply applied to any state-of-art algorithms.

Data compression is becoming critical for storing scientific data because many scientific applications need to store large amounts of data and post process this data for scientific discovery. Unlike image and video compression algorithms that limit errors to primary data, scientists require compression techniques that accurately preserve derived quantities of interest (QoIs). This paper presents a physics-informed compression technique implemented as an end-to-end, scalable, GPU-based pipeline for data compression that addresses this requirement. Our hybrid compression technique combines machine learning techniques and standard compression methods. Specifically, we combine an autoencoder, an error-bounded lossy compressor to provide guarantees on raw data error, and a constraint satisfaction post-processing step to preserve the QoIs within a minimal error (generally less than floating point error). The effectiveness of the data compression pipeline is demonstrated by compressing nuclear fusion simulation data generated by a large-scale fusion code, XGC, which produces hundreds of terabytes of data in a single day. Our approach works within the ADIOS framework and results in compression by a factor of more than 150 while requiring only a few percent of the computational resources necessary for generating the data, making the overall approach highly effective for practical scenarios.

Task-oriented dialogue (TOD) systems are mainly based on the slot-filling-based TOD (SF-TOD) framework, in which dialogues are broken down into smaller, controllable units (i.e., slots) to fulfill a specific task. A series of approaches based on this framework achieved remarkable success on various TOD benchmarks. However, we argue that the current TOD benchmarks are limited to surrogate real-world scenarios and that the current TOD models are still a long way from unraveling the scenarios. In this position paper, we first identify current status and limitations of SF-TOD systems. After that, we explore the WebTOD framework, the alternative direction for building a scalable TOD system when a web/mobile interface is available. In WebTOD, the dialogue system learns how to understand the web/mobile interface that the human agent interacts with, powered by a large-scale language model.

Fusion-in-Decoder (FiD) is a powerful retrieval-augmented language model that sets the state-of-the-art on many knowledge-intensive NLP tasks. However, FiD suffers from very expensive inference. We show that the majority of inference time results from memory bandwidth constraints in the decoder, and propose two simple changes to the FiD architecture to speed up inference by 7x. The faster decoder inference then allows for a much larger decoder. We denote FiD with the above modifications as FiDO, and show that it strongly improves performance over existing FiD models for a wide range of inference budgets. For example, FiDO-Large-XXL performs faster inference than FiD-Base and achieves better performance than FiD-Large.